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What Is OpenStack? 


o open-source cloud computing platform 
o created by Rackspace and NASA in 2010 
o written in python 


o modular and microservices architecture 


o used for a public/private cloud 
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Why do we need OpenStack? 


© open-source self-hosted solution for private / public clouds 
o Vmware alternative with zero price tag 


o a strong network isolation 
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- Command-line interfaces (nova, neutron, swift, and so on) 
- Cloud Management Tools (Rightseale, Enstratius, and so on.) 
- GUI tools (Dashboard, Cyberduck, Phone client, and so on.) 
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What's the problem? 


o hundreds of microservices 
o hundreds of bare-metal servers 


a huge python codebase 


O 


o a full update at least twice a year (upstream release period) 
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Why Kubernetes? 


V built to manage thousands of microservices 

V can scale to hundreds of nodes 

V containerization solves the problem with dependencies 
V self-healing, high availability, healthchecks 


V and many other benefits ... 
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But ... 


ONE DOES NOT SIMPLY OpenStack is not just an application 


VMs will be running on k8s workers 
OpenStack has its own network stack 


a complicated order of starting services 


INSTALL OPENSTACK TO K&S 


a storage based on Ceph 
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Moreover 


O ~ 


Controller Node Network Node Compute Node Storage Node 
——— 
3) 


Public (neutron external interface) 


Administration 


Management/API (api interface) 
Virtual machine (tunnel interface) 


Storage (storage interface) 


KAK 
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And how does K8s help here? 


General Tips 


do not reinvent the wheel 


use openstack-helm 
use official docker images when possible 
run all the OpenStack services in one namespace 


RTFM (if you can find it) 
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Database 


o Percona XtraDB Cluster with k8s operator 
o separate database cluster for Neutron (network system) 
o use fast SSDs if cluster > 50 compute nodes 


o monitoring 
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Storage system 


O Ceph Fs is the most popular 
o one Ceph cluster for k8s and for OpenStack (different pools) 
o a separate physical network for a storage 
o dedicated storage hosts if you have the budget: 
+ to reduce load 
+ to reduce chances of losing data 


+ to have faster reboot of compute nodes 
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How OS network works 


o SDN OpenvSwitch / OVN 


Network node 
Layer 3 Router 


(neutron-|3-agent) 


L2: VXLAN / Geneve / VLAN 


O 


o L3: virtual routers / OVN 
dnsmasq DHCP / DNS 


O 


Compute node Compute node 


o service called “Neutron” 


— Layer 2 Switch Layer 2 Switch 
(neutron-openvswitch-agent) (neutron-openvswitch-agent) 
Instance Instance 
192.168.200.15/24 192.168.200.16/24 
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Network challenges 


1. How does OpenvSwitch/OVN configure host system? 


Control Cluster 


ovsdb-server * ovs-vswitchd 


Management Protocol (6632/TCP) 
dr OpenFlow (6633/TCP) 
s Netlink 


OVS Kernel Module 
— 


16 CLOUDIFICATIONN 


Network challenges 
/ 
/ 
/ 
2. External networks only VLAN based / 
Management Network k 

i 

| 

| 

| 

| 

| 

| 

| 

Network Node | Traffic to 

Compute Node | unknown target 

| 

Data Net (VxLAN) | 

Cloud Controller | bl 
WI 
External Net (VLAN) API Network — | FINE 
_— ony | Unknown 
/ 27 unicast flood 
/ NA 
/ 
/ 
/ 
/ AN 

CLOUDIFICATION 


17 


Network Tips: OVS 


spec: 
hostNetwork: true 


containers: 


1. OpenvSwitch daemon: securityContext: 
capabilities: 
add: 
o host network - NET ADMIN 
- SYS MODULE 
- SYS NICE 


o Capabilities 


readOnlyRootFilesystem: true 
runAsUser: 0 


O run as root 


volume Mounts: 
- mountPath: / run 
8 name: run 
O mount /run directory from the S 
volumes: 
- hostPath: 
path: /run 
type: Directory 
name: run 


host system 
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Network Tips: S&L TOR VLANs 


2. External networks: 


Layer 3 
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VLAN 100 VLAN 300 


Layer 2 


/VLAN 206 I VLAN 200 
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Network Tips: S&L without VLANs 


2. External networks: 


Layer 3 (BGP) 


_.- Leaf switch 
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Network Tips: solutions 


use segments extension and per- 
rack VLANs 

use BGP dynamic routing plugin 
use DVR routers when it’s possible 
use EVPN-VXLAN 


network in the data center 
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Compute 


o Nova configures KVM on the host system 


o VM can have a direct access for network/GPU cards 


— — 


O Privileged libvirt container 


o Mounts from the host system: 


> /lib/modules 
> /[var/lib/nova 
> /var/lib/libvirt 
/run 
> /sys/fs/cgroups 
AN 
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Compute 


o State directories with RW access from all the hosts for the migrations 


# k describe pvc -n openstack var-lib-nova 


Name: var-lib-nova 
Namespace: openstack 
spec. StorageClass: cephfs 
securityContext: Access Modes: RWX 
privileged: true VolumeMode: Filesystem 


Mounted By: libvirt-libvirt-default-5v2ld 
libvirt-libvirt-default-7ckld 
libvirt-libvirt-default-7prvw 
libvirt-libvirt-default-f2vpp 


- mountPath: /var/lib/nova-cephfs 
name: cephfs-var-lib-nova 


volumes: 


libvirt-libvirt-default-sq5rs 
- name: cephfs-var-lib-nova nova-compute-default-2kmhd 
persistentVolumeClaim: nova-compute-default-4cpf8 
claimName: var-lib-nova nova-compute-default-728lq 


nova-compute-default-r5lk7 
nova-compute-default-rtxkm 
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Is OpenStack ready? 
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o No graceful restart 


neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 
neutron. 


o Bad or non-existent healthchecks 


o Multiline logs (no json support!) 


„agent Traceback (most recent call last): 
‚agent 
. agent 
. agent 
.agent 
agent 
. agent 
.agent 
.agent 
agent 
agent 
. agent 
agent 
agent os logmessaging. rpc. client. Remote ET ro: Remote error: Internal Error (pymysq lb. err. Internal Error) (1047, 'WSREP has not ye 
„agent (Background on this error at: http: // sqblal che. me /e / 285) 


File / var / lib / openstack / Lib / python3. 6 s te-packages / neutron / agent / dhcp / agent. py“, Line 1044, in report state 
ctx, self. agent state, True) 

File %/var / lib / openstack / Lib / python3. 6 ste- packages / neut ron / agent / pC. pY“, Line 101, in report state 
return method( context, report state“, **kwargs) 

File % var / Lib / openstack / Lib / python3. 6/ site -packages / os lormessaging/ rpc / client. py“, Line 181, in call 
transport options=self.transport options) 

File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in send 
transport options=transport options) 

File "/var/Llib/openstack/Llib/python3.6/site-packages/oslo messaging/ drivers/amqpdriver.py", Line 682, in send 
transport options=transport options) 

File "/var/Llib/openstack/Llib/python3.6/site-packages/oslo messaging/ drivers/amqpdriver.py", line 672, in send 
raise result 
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Armenia 


Is OpenStack ready? 


o Bad monitoring abilities 
o Complex dependencies between components 


o Difficult to customize images with components 
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If everything is so bad, why K8s? 


o Anyway, it gives better control over hundreds of services with K8s 
o It gives more stability with updates 

o Self-healing, HA, isolation, etc. 

o It’s easier to control at a large scale 


o K8s is more popular than OpenStack 
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feedback on what 
you've liked or what 
could be improved 


You can rate the talk and 
give 


Leave your feedback! 
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C) https://github.com/velp 


